[jira] [Resolved] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16173. Resolution: Fixed Fix Version/s: 1.6.2 Issue resolved by pull request 13902

[jira] [Resolved] (SPARK-16186) Support partition batch pruning with `IN` predicate in InMemoryTableScanExec

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16186. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13887

[jira] [Resolved] (SPARK-16077) Python UDF may fail because of six

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16077. Resolution: Fixed Fix Version/s: 2.0.1 1.6.3 Issue resolved by pull

[jira] [Assigned] (SPARK-16077) Python UDF may fail because of six

2016-06-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-16077: -- Assignee: Davies Liu > Python UDF may fail because of six >

[jira] [Assigned] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-16179: -- Assignee: Davies Liu > UDF explosion yielding empty dataframe fails >

[jira] [Updated] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16179: --- Affects Version/s: 2.0.0 > UDF explosion yielding empty dataframe fails >

[jira] [Updated] (SPARK-16180) Task hang on fetching blocks (cached RDD)

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16180: --- Description: Here is the stackdump of executor: {code} sun.misc.Unsafe.park(Native Method)

[jira] [Updated] (SPARK-16180) Task hang on fetching blocks (cached RDD)

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16180: --- Affects Version/s: 1.6.1 > Task hang on fetching blocks (cached RDD) >

[jira] [Created] (SPARK-16180) Task hang on fetching blocks (cached RDD)

2016-06-23 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16180: -- Summary: Task hang on fetching blocks (cached RDD) Key: SPARK-16180 URL: https://issues.apache.org/jira/browse/SPARK-16180 Project: Spark Issue Type:

[jira] [Updated] (SPARK-16175) Handle None for all Python UDT

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16175: --- Affects Version/s: 2.0.0 1.6.1 > Handle None for all Python UDT >

[jira] [Created] (SPARK-16175) Handle None for all Python UDT

2016-06-23 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16175: -- Summary: Handle None for all Python UDT Key: SPARK-16175 URL: https://issues.apache.org/jira/browse/SPARK-16175 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-16163) Statistics of logical plan is super slow on large query

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347136#comment-15347136 ] Davies Liu commented on SPARK-16163: [~srowen] yes, thanks for correct it. > Statistics of logical

[jira] [Created] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-23 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16173: -- Summary: Can't join describe() of DataFrame in Scala 2.10 Key: SPARK-16173 URL: https://issues.apache.org/jira/browse/SPARK-16173 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-16163) Statistics of logical plan is super slow on large query

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16163. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13871

[jira] [Updated] (SPARK-16163) Statistics of logical plan is super slow on large query

2016-06-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16163: --- Affects Version/s: 2.0.0 > Statistics of logical plan is super slow on large query >

[jira] [Created] (SPARK-16163) Statistics of logical plan is super slow on large query

2016-06-22 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16163: -- Summary: Statistics of logical plan is super slow on large query Key: SPARK-16163 URL: https://issues.apache.org/jira/browse/SPARK-16163 Project: Spark Issue

[jira] [Resolved] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16003. Resolution: Fixed Fix Version/s: 2.0.0 > SerializationDebugger run into infinite loop >

[jira] [Updated] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16003: --- Assignee: Eric Liang > SerializationDebugger run into infinite loop >

[jira] [Updated] (SPARK-16104) Do not creaate CSV writer object for every flush when writing

2016-06-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16104: --- Assignee: Hyukjin Kwon > Do not creaate CSV writer object for every flush when writing >

[jira] [Resolved] (SPARK-16104) Do not creaate CSV writer object for every flush when writing

2016-06-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16104. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13809

[jira] [Resolved] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16086. Resolution: Fixed Fix Version/s: (was: 1.6.2) (was: 1.5.3)

[jira] [Commented] (SPARK-16077) Python UDF may fail because of six

2016-06-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342262#comment-15342262 ] Davies Liu commented on SPARK-16077: [~bill_chambers] We fixed it for some cases, could still fail in

[jira] [Resolved] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16086. Resolution: Fixed Fix Version/s: 1.6.2 1.5.3 2.0.0

[jira] [Updated] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16086: --- Description: {code} >>> sqlContext.registerFunction("f", lambda : "a") >>> sqlContext.sql("select

[jira] [Updated] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16086: --- Affects Version/s: 1.5.2 > Python UDF failed when there is no arguments >

[jira] [Created] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-20 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16086: -- Summary: Python UDF failed when there is no arguments Key: SPARK-16086 URL: https://issues.apache.org/jira/browse/SPARK-16086 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-16078) from_utc_timestamp/to_utc_timestamp may give different result in different timezone

2016-06-20 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16078: -- Summary: from_utc_timestamp/to_utc_timestamp may give different result in different timezone Key: SPARK-16078 URL: https://issues.apache.org/jira/browse/SPARK-16078

[jira] [Created] (SPARK-16077) Python UDF may fail because of six

2016-06-20 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16077: -- Summary: Python UDF may fail because of six Key: SPARK-16077 URL: https://issues.apache.org/jira/browse/SPARK-16077 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-15613) Incorrect days to millis conversion

2016-06-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15613. Resolution: Fixed Fix Version/s: 1.6.2 2.0.0 Issue resolved by pull

[jira] [Updated] (SPARK-15803) Support with statement syntax for SparkSession

2016-06-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15803: --- Assignee: Jeff Zhang > Support with statement syntax for SparkSession >

[jira] [Resolved] (SPARK-15803) Support with statement syntax for SparkSession

2016-06-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15803. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13541

[jira] [Commented] (SPARK-13753) Column nullable is derived incorrectly

2016-06-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336567#comment-15336567 ] Davies Liu commented on SPARK-13753: After discussed with [~cloud_fan], we do have runtime check to

[jira] [Created] (SPARK-16011) SQL metrics include duplicated attempts

2016-06-16 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16011: -- Summary: SQL metrics include duplicated attempts Key: SPARK-16011 URL: https://issues.apache.org/jira/browse/SPARK-16011 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-16 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15822. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13723

[jira] [Updated] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-16 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16003: --- Description: This is observed while debugging https://issues.apache.org/jira/browse/SPARK-15811

[jira] [Updated] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-16 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16003: --- Description: This is observed while debugging https://issues.apache.org/jira/browse/SPARK-15811

[jira] [Created] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-16 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16003: -- Summary: SerializationDebugger run into infinite loop Key: SPARK-16003 URL: https://issues.apache.org/jira/browse/SPARK-16003 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-15811) Python UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-16 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Description: I've built spark-2.0-preview (8f5a04b) with scala-2.10 using the following {code}

[jira] [Resolved] (SPARK-15934) Return binary mode in ThriftServer

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15934. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13667

[jira] [Resolved] (SPARK-15888) Python UDF over aggregate fails

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15888. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13682

[jira] [Updated] (SPARK-15811) Python UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Description: I've built spark-2.0-preview (8f5a04b) with scala-2.10 using the following {code}

[jira] [Updated] (SPARK-15811) Python UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Summary: Python UDFs do not work in Spark 2.0-preview built with scala 2.10 (was: UDFs do not work

[jira] [Assigned] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15811: -- Assignee: Davies Liu > UDFs do not work in Spark 2.0-preview built with scala 2.10 >

[jira] [Updated] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Priority: Blocker (was: Critical) > UDFs do not work in Spark 2.0-preview built with scala 2.10 >

[jira] [Updated] (SPARK-15888) Python UDF over aggregate fails

2016-06-14 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15888: --- Priority: Blocker (was: Major) > Python UDF over aggregate fails > ---

[jira] [Assigned] (SPARK-15888) Python UDF over aggregate fails

2016-06-14 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15888: -- Assignee: Davies Liu > Python UDF over aggregate fails > --- > >

[jira] [Assigned] (SPARK-15613) Incorrect days to millis conversion

2016-06-13 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15613: -- Assignee: Davies Liu > Incorrect days to millis conversion >

[jira] [Created] (SPARK-15896) Clean shuffle files after finish the SQL query

2016-06-11 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15896: -- Summary: Clean shuffle files after finish the SQL query Key: SPARK-15896 URL: https://issues.apache.org/jira/browse/SPARK-15896 Project: Spark Issue Type:

[jira] [Commented] (SPARK-15888) Python UDF over aggregate fails

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325733#comment-15325733 ] Davies Liu commented on SPARK-15888: After some investigation, it turned out to be that the Python

[jira] [Updated] (SPARK-15888) Python UDF over aggregate fails

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15888: --- Summary: Python UDF over aggregate fails (was: UDF fails in Python) > Python UDF over aggregate

[jira] [Resolved] (SPARK-15759) Fallback to non-codegen if fail to compile generated code

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15759. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13501

[jira] [Updated] (SPARK-15678) Not use cache on appends and overwrites

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15678: --- Assignee: Sameer Agarwal > Not use cache on appends and overwrites >

[jira] [Resolved] (SPARK-15678) Not use cache on appends and overwrites

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15678. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13566

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325417#comment-15325417 ] Davies Liu commented on SPARK-15822: The latest stacktrace is different than previous one, it seems

[jira] [Resolved] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15654. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13531

[jira] [Resolved] (SPARK-15825) sort-merge-join gives invalid results when joining on a tupled key

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15825. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13589

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324907#comment-15324907 ] Davies Liu commented on SPARK-15822: SortMergeJoin assume that the keys do not have null in them (we

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324875#comment-15324875 ] Davies Liu commented on SPARK-15822: Could you try to disable whole-stage codegen to see whether this

[jira] [Updated] (SPARK-15433) PySpark core test should not use SerDe from PythonMLLibAPI

2016-06-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15433: --- Assignee: Liang-Chi Hsieh > PySpark core test should not use SerDe from PythonMLLibAPI >

[jira] [Updated] (SPARK-14670) Allow updating SQLMetrics on driver

2016-06-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14670: --- Assignee: Wenchen Fan (was: Andrew Or) > Allow updating SQLMetrics on driver >

[jira] [Resolved] (SPARK-14670) Allow updating SQLMetrics on driver

2016-06-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14670. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13189

[jira] [Updated] (SPARK-15791) NPE in ScalarSubquery

2016-06-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15791: --- Assignee: Eric Liang (was: Davies Liu) > NPE in ScalarSubquery > - > >

[jira] [Created] (SPARK-15791) NPE in ScalarSubquery

2016-06-06 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15791: -- Summary: NPE in ScalarSubquery Key: SPARK-15791 URL: https://issues.apache.org/jira/browse/SPARK-15791 Project: Spark Issue Type: Bug Components: SQL

[jira] [Assigned] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15654: -- Assignee: Davies Liu (was: Takeshi Yamamuro) > Reading gzipped files results in duplicate

[jira] [Resolved] (SPARK-15391) Spark executor OOM during TimSort

2016-06-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15391. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13318

[jira] [Created] (SPARK-15759) Fallback to non-codegen if fail to compile generated code

2016-06-03 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15759: -- Summary: Fallback to non-codegen if fail to compile generated code Key: SPARK-15759 URL: https://issues.apache.org/jira/browse/SPARK-15759 Project: Spark Issue

[jira] [Resolved] (SPARK-15671) performance regression CoalesceRDD large # partitions

2016-06-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15671. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13443

[jira] [Updated] (SPARK-15671) performance regression CoalesceRDD large # partitions

2016-06-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15671: --- Assignee: Thomas Graves > performance regression CoalesceRDD large # partitions >

[jira] [Updated] (SPARK-15557) expression ((cast(99 as decimal) + '3') * '2.3' ) return null

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15557: --- Assignee: Dilip Biswal > expression ((cast(99 as decimal) + '3') * '2.3' ) return null >

[jira] [Resolved] (SPARK-15557) expression ((cast(99 as decimal) + '3') * '2.3' ) return null

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15557. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13368

[jira] [Resolved] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15327. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13235

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308673#comment-15308673 ] Davies Liu commented on SPARK-11293: I think your patch is not related to this bug, right? >

[jira] [Assigned] (SPARK-11293) Spillable collections leak shuffle memory

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11293: -- Assignee: Davies Liu > Spillable collections leak shuffle memory >

[jira] [Updated] (SPARK-15568) TimSort and RadixSort can't support more than 1 billions elements

2016-05-26 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15568: --- Summary: TimSort and RadixSort can't support more than 1 billions elements (was: TimSort and

[jira] [Created] (SPARK-15568) TimSort and RadixSort can't support more than 2 billions elements

2016-05-26 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15568: -- Summary: TimSort and RadixSort can't support more than 2 billions elements Key: SPARK-15568 URL: https://issues.apache.org/jira/browse/SPARK-15568 Project: Spark

[jira] [Commented] (SPARK-15554) Duplicated executors in Spark UI

2016-05-26 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301627#comment-15301627 ] Davies Liu commented on SPARK-15554: cc [~zsxwing] > Duplicated executors in Spark UI >

[jira] [Created] (SPARK-15554) Duplicated executors in Spark UI

2016-05-26 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15554: -- Summary: Duplicated executors in Spark UI Key: SPARK-15554 URL: https://issues.apache.org/jira/browse/SPARK-15554 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-15332) OutOfMemory in TimSort

2016-05-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15332: -- Assignee: Davies Liu > OutOfMemory in TimSort > --- > >

[jira] [Assigned] (SPARK-15391) Spark executor OOM during TimSort

2016-05-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15391: -- Assignee: Davies Liu > Spark executor OOM during TimSort > -

[jira] [Resolved] (SPARK-12795) Whole stage codegen

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12795. Resolution: Fixed Fix Version/s: 2.0.0 > Whole stage codegen > --- > >

[jira] [Closed] (SPARK-14748) BoundReference should not set ExprCode.code to empty string

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-14748. -- Resolution: Won't Fix Won't fix this for now. > BoundReference should not set ExprCode.code to empty

[jira] [Updated] (SPARK-12949) Support common expression elimination

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12949: --- Assignee: Liang-Chi Hsieh > Support common expression elimination >

[jira] [Resolved] (SPARK-12949) Support common expression elimination

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12949. Resolution: Fixed Fix Version/s: 2.0.0 > Support common expression elimination >

[jira] [Commented] (SPARK-12949) Support common expression elimination

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298537#comment-15298537 ] Davies Liu commented on SPARK-12949: Common subexpress elimination in Aggregate are supported by

[jira] [Updated] (SPARK-13135) Don't print expressions recursively in generated code

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13135: --- Assignee: Dongjoon Hyun > Don't print expressions recursively in generated code >

[jira] [Resolved] (SPARK-13135) Don't print expressions recursively in generated code

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13135. Resolution: Fixed Target Version/s: 2.0.0 > Don't print expressions recursively in

[jira] [Resolved] (SPARK-15433) PySpark core test should not use SerDe from PythonMLLibAPI

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15433. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13214

[jira] [Updated] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

2016-05-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14343: --- Priority: Blocker (was: Critical) > Dataframe operations on a partitioned dataset (using partition

[jira] [Commented] (SPARK-14946) Spark 2.0 vs 1.6.1 Query Time(out)

2016-05-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296697#comment-15296697 ] Davies Liu commented on SPARK-14946: [~raymond.honderd...@sizmek.com] Thanks for the feedback, I'm

[jira] [Updated] (SPARK-15441) dataset outer join seems to return incorrect result

2016-05-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15441: --- Assignee: Wenchen Fan > dataset outer join seems to return incorrect result >

[jira] [Commented] (SPARK-15441) dataset outer join seems to return incorrect result

2016-05-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294807#comment-15294807 ] Davies Liu commented on SPARK-15441: How to we represent a null in Dataset? If it's a row with all

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294785#comment-15294785 ] Davies Liu commented on SPARK-15285: [~kiszk] Go ahead, don't know why I can't assign this to you. >

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15285: --- Assignee: (was: Wenchen Fan) > Generated SpecificSafeProjection.apply method grows beyond 64 KB

[jira] [Resolved] (SPARK-15078) Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15078. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13188

[jira] [Assigned] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15327: -- Assignee: Davies Liu > Catalyst code generation fails with complex data structure >

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294178#comment-15294178 ] Davies Liu commented on SPARK-15285: cc [~cloud_fan] > Generated SpecificSafeProjection.apply method

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15285: --- Assignee: Wenchen Fan > Generated SpecificSafeProjection.apply method grows beyond 64 KB >

[jira] [Commented] (SPARK-14331) Exceptions saving to parquetFile after join from dataframes in master

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294168#comment-15294168 ] Davies Liu commented on SPARK-14331: Could you post the full stacktrace? This exception should be

[jira] [Closed] (SPARK-15448) Flaky test:pyspark.ml.tests.DefaultValuesTests.test_java_params

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-15448. -- Resolution: Duplicate Fix Version/s: 2.0.0 > Flaky

[jira] [Assigned] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-14031: -- Assignee: Davies Liu > Dataframe to csv IO, system performance enters high CPU state and

<    1   2   3   4   5   6   7   8   9   10   >