[jira] [Created] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-23 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16173: -- Summary: Can't join describe() of DataFrame in Scala 2.10 Key: SPARK-16173 URL: https://issues.apache.org/jira/browse/SPARK-16173 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-16163) Statistics of logical plan is super slow on large query

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16163. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13871 [https://github.

[jira] [Updated] (SPARK-16163) Statistics of logical plan is super slow on large query

2016-06-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16163: --- Affects Version/s: 2.0.0 > Statistics of logical plan is super slow on large query >

[jira] [Created] (SPARK-16163) Statistics of logical plan is super slow on large query

2016-06-22 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16163: -- Summary: Statistics of logical plan is super slow on large query Key: SPARK-16163 URL: https://issues.apache.org/jira/browse/SPARK-16163 Project: Spark Issue Typ

[jira] [Resolved] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16003. Resolution: Fixed Fix Version/s: 2.0.0 > SerializationDebugger run into infinite loop >

[jira] [Updated] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16003: --- Assignee: Eric Liang > SerializationDebugger run into infinite loop > ---

[jira] [Updated] (SPARK-16104) Do not creaate CSV writer object for every flush when writing

2016-06-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16104: --- Assignee: Hyukjin Kwon > Do not creaate CSV writer object for every flush when writing >

[jira] [Resolved] (SPARK-16104) Do not creaate CSV writer object for every flush when writing

2016-06-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16104. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13809 [https://github.

[jira] [Resolved] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16086. Resolution: Fixed Fix Version/s: (was: 1.6.2) (was: 1.5.3)

[jira] [Commented] (SPARK-16077) Python UDF may fail because of six

2016-06-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342262#comment-15342262 ] Davies Liu commented on SPARK-16077: [~bill_chambers] We fixed it for some cases, cou

[jira] [Resolved] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16086. Resolution: Fixed Fix Version/s: 1.6.2 1.5.3 2.0.0 Iss

[jira] [Updated] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16086: --- Description: {code} >>> sqlContext.registerFunction("f", lambda : "a") >>> sqlContext.sql("select f()

[jira] [Updated] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16086: --- Affects Version/s: 1.5.2 > Python UDF failed when there is no arguments > ---

[jira] [Created] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-20 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16086: -- Summary: Python UDF failed when there is no arguments Key: SPARK-16086 URL: https://issues.apache.org/jira/browse/SPARK-16086 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-16078) from_utc_timestamp/to_utc_timestamp may give different result in different timezone

2016-06-20 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16078: -- Summary: from_utc_timestamp/to_utc_timestamp may give different result in different timezone Key: SPARK-16078 URL: https://issues.apache.org/jira/browse/SPARK-16078 Proje

[jira] [Created] (SPARK-16077) Python UDF may fail because of six

2016-06-20 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16077: -- Summary: Python UDF may fail because of six Key: SPARK-16077 URL: https://issues.apache.org/jira/browse/SPARK-16077 Project: Spark Issue Type: Bug Comp

[jira] [Resolved] (SPARK-15613) Incorrect days to millis conversion

2016-06-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15613. Resolution: Fixed Fix Version/s: 1.6.2 2.0.0 Issue resolved by pull reque

[jira] [Updated] (SPARK-15803) Support with statement syntax for SparkSession

2016-06-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15803: --- Assignee: Jeff Zhang > Support with statement syntax for SparkSession > -

[jira] [Resolved] (SPARK-15803) Support with statement syntax for SparkSession

2016-06-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15803. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13541 [https://github.

[jira] [Commented] (SPARK-13753) Column nullable is derived incorrectly

2016-06-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15336567#comment-15336567 ] Davies Liu commented on SPARK-13753: After discussed with [~cloud_fan], we do have ru

[jira] [Comment Edited] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-17 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335369#comment-15335369 ] Jinxia Liu edited comment on SPARK-12177 at 6/17/16 7:27 AM: -

[jira] [Created] (SPARK-16011) SQL metrics include duplicated attempts

2016-06-16 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16011: -- Summary: SQL metrics include duplicated attempts Key: SPARK-16011 URL: https://issues.apache.org/jira/browse/SPARK-16011 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-16 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15822. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13723 [https://github.

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-16 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335369#comment-15335369 ] Jinxia Liu commented on SPARK-12177: Thanks Cody, I will check your example. I can

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-16 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335363#comment-15335363 ] Jinxia Liu commented on SPARK-12177: Okay if you think its ok then let it be. > Upda

[jira] [Comment Edited] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-16 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335200#comment-15335200 ] Jinxia Liu edited comment on SPARK-12177 at 6/17/16 2:39 AM: -

[jira] [Comment Edited] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-16 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335215#comment-15335215 ] Jinxia Liu edited comment on SPARK-12177 at 6/17/16 2:03 AM: -

[jira] [Comment Edited] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-16 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335200#comment-15335200 ] Jinxia Liu edited comment on SPARK-12177 at 6/17/16 2:03 AM: -

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-16 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335215#comment-15335215 ] Jinxia Liu commented on SPARK-12177: Hi Cody, should the log usage be unified in the

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-16 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335200#comment-15335200 ] Jinxia Liu commented on SPARK-12177: Hi Cody, I did some tests today using your conn

[jira] [Updated] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-16 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16003: --- Description: This is observed while debugging https://issues.apache.org/jira/browse/SPARK-15811 {co

[jira] [Updated] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-16 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16003: --- Description: This is observed while debugging https://issues.apache.org/jira/browse/SPARK-15811 {co

[jira] [Created] (SPARK-16003) SerializationDebugger run into infinite loop

2016-06-16 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16003: -- Summary: SerializationDebugger run into infinite loop Key: SPARK-16003 URL: https://issues.apache.org/jira/browse/SPARK-16003 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-15811) Python UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-16 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Description: I've built spark-2.0-preview (8f5a04b) with scala-2.10 using the following {code} ./dev

[jira] [Resolved] (SPARK-15934) Return binary mode in ThriftServer

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15934. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13667 [https://github.

[jira] [Resolved] (SPARK-15888) Python UDF over aggregate fails

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15888. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13682 [https://github.

[jira] [Updated] (SPARK-15811) Python UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Description: I've built spark-2.0-preview (8f5a04b) with scala-2.10 using the following {code} ./dev

[jira] [Updated] (SPARK-15969) FileNotFoundException: Multiple arguments for py-files flag, (also jars) for spark-submit

2016-06-15 Thread Kun Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kun Liu updated SPARK-15969: Remaining Estimate: 120h (was: 168h) Original Estimate: 120h (was: 168h) > FileNotFoundException: Mu

[jira] [Updated] (SPARK-15811) Python UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Summary: Python UDFs do not work in Spark 2.0-preview built with scala 2.10 (was: UDFs do not work i

[jira] [Created] (SPARK-15969) FileNotFoundException: Multiple arguments for py-files flag, (also jars) for spark-submit

2016-06-15 Thread Kun Liu (JIRA)
Kun Liu created SPARK-15969: --- Summary: FileNotFoundException: Multiple arguments for py-files flag, (also jars) for spark-submit Key: SPARK-15969 URL: https://issues.apache.org/jira/browse/SPARK-15969 Proje

[jira] [Assigned] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15811: -- Assignee: Davies Liu > UDFs do not work in Spark 2.0-preview built with scala 2.10 > -

[jira] [Updated] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Priority: Blocker (was: Critical) > UDFs do not work in Spark 2.0-preview built with scala 2.10 > --

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-14 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331168#comment-15331168 ] Jinxia Liu commented on SPARK-12177: Thanks Cody! > Update KafkaDStreams to new Kafk

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-14 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331026#comment-15331026 ] Jinxia Liu commented on SPARK-12177: [~c...@koeninger.org] thanks for the quick reply

[jira] [Comment Edited] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-14 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331007#comment-15331007 ] Jinxia Liu edited comment on SPARK-12177 at 6/15/16 2:29 AM: -

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-14 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331007#comment-15331007 ] Jinxia Liu commented on SPARK-12177: [~c...@koeninger.org] thanks for contributing th

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-14 Thread Jinxia Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331005#comment-15331005 ] Jinxia Liu commented on SPARK-12177: [~c...@koeninger.org] thanks for contributing th

[jira] [Commented] (SPARK-3451) spark-submit should support specifying glob wildcards in the --jars CLI option

2016-06-14 Thread Kun Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330945#comment-15330945 ] Kun Liu commented on SPARK-3451: Vote for this feature. If no one will, I may try as my f

[jira] [Updated] (SPARK-15888) Python UDF over aggregate fails

2016-06-14 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15888: --- Priority: Blocker (was: Major) > Python UDF over aggregate fails > --- >

[jira] [Assigned] (SPARK-15888) Python UDF over aggregate fails

2016-06-14 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15888: -- Assignee: Davies Liu > Python UDF over aggregate fails > --- > >

[jira] [Assigned] (SPARK-15613) Incorrect days to millis conversion

2016-06-13 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15613: -- Assignee: Davies Liu > Incorrect days to millis conversion >

[jira] [Created] (SPARK-15896) Clean shuffle files after finish the SQL query

2016-06-10 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15896: -- Summary: Clean shuffle files after finish the SQL query Key: SPARK-15896 URL: https://issues.apache.org/jira/browse/SPARK-15896 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-15888) Python UDF over aggregate fails

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325733#comment-15325733 ] Davies Liu commented on SPARK-15888: After some investigation, it turned out to be th

[jira] [Updated] (SPARK-15888) Python UDF over aggregate fails

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15888: --- Summary: Python UDF over aggregate fails (was: UDF fails in Python) > Python UDF over aggregate fail

[jira] [Resolved] (SPARK-15759) Fallback to non-codegen if fail to compile generated code

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15759. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13501 [https://github.

[jira] [Updated] (SPARK-15678) Not use cache on appends and overwrites

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15678: --- Assignee: Sameer Agarwal > Not use cache on appends and overwrites >

[jira] [Resolved] (SPARK-15678) Not use cache on appends and overwrites

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15678. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13566 [https://github.

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325417#comment-15325417 ] Davies Liu commented on SPARK-15822: The latest stacktrace is different than previous

[jira] [Resolved] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15654. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13531 [https://github.

[jira] [Resolved] (SPARK-15825) sort-merge-join gives invalid results when joining on a tupled key

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15825. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13589 [https://github.

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324907#comment-15324907 ] Davies Liu commented on SPARK-15822: SortMergeJoin assume that the keys do not have n

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324875#comment-15324875 ] Davies Liu commented on SPARK-15822: Could you try to disable whole-stage codegen to

[jira] [Updated] (SPARK-15433) PySpark core test should not use SerDe from PythonMLLibAPI

2016-06-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15433: --- Assignee: Liang-Chi Hsieh > PySpark core test should not use SerDe from PythonMLLibAPI >

[jira] [Updated] (SPARK-14670) Allow updating SQLMetrics on driver

2016-06-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14670: --- Assignee: Wenchen Fan (was: Andrew Or) > Allow updating SQLMetrics on driver > -

[jira] [Resolved] (SPARK-14670) Allow updating SQLMetrics on driver

2016-06-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14670. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13189 [https://github.

[jira] [Updated] (SPARK-15791) NPE in ScalarSubquery

2016-06-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15791: --- Assignee: Eric Liang (was: Davies Liu) > NPE in ScalarSubquery > - > >

[jira] [Created] (SPARK-15791) NPE in ScalarSubquery

2016-06-06 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15791: -- Summary: NPE in ScalarSubquery Key: SPARK-15791 URL: https://issues.apache.org/jira/browse/SPARK-15791 Project: Spark Issue Type: Bug Components: SQL

[jira] [Assigned] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15654: -- Assignee: Davies Liu (was: Takeshi Yamamuro) > Reading gzipped files results in duplicate row

[jira] [Resolved] (SPARK-15391) Spark executor OOM during TimSort

2016-06-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15391. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13318 [https://github.

[jira] [Created] (SPARK-15759) Fallback to non-codegen if fail to compile generated code

2016-06-03 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15759: -- Summary: Fallback to non-codegen if fail to compile generated code Key: SPARK-15759 URL: https://issues.apache.org/jira/browse/SPARK-15759 Project: Spark Issue T

[jira] [Resolved] (SPARK-15671) performance regression CoalesceRDD large # partitions

2016-06-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15671. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13443 [https://github.

[jira] [Updated] (SPARK-15671) performance regression CoalesceRDD large # partitions

2016-06-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15671: --- Assignee: Thomas Graves > performance regression CoalesceRDD large # partitions > ---

[jira] [Updated] (SPARK-15557) expression ((cast(99 as decimal) + '3') * '2.3' ) return null

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15557: --- Assignee: Dilip Biswal > expression ((cast(99 as decimal) + '3') * '2.3' ) return null >

[jira] [Resolved] (SPARK-15557) expression ((cast(99 as decimal) + '3') * '2.3' ) return null

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15557. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13368 [https://github.

[jira] [Resolved] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15327. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13235 [https://github.

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308673#comment-15308673 ] Davies Liu commented on SPARK-11293: I think your patch is not related to this bug, r

[jira] [Assigned] (SPARK-11293) Spillable collections leak shuffle memory

2016-05-31 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11293: -- Assignee: Davies Liu > Spillable collections leak shuffle memory > ---

[jira] [Comment Edited] (SPARK-15641) Incorrect Completed for Incomplete applications in HistoryServer

2016-05-29 Thread Zhuo Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306021#comment-15306021 ] Zhuo Liu edited comment on SPARK-15641 at 5/29/16 6:31 PM: --- [~s

[jira] [Commented] (SPARK-15641) Incorrect Completed for Incomplete applications in HistoryServer

2016-05-29 Thread Zhuo Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306021#comment-15306021 ] Zhuo Liu commented on SPARK-15641: -- [~sowen], thanks Sean for the reminder, that is true

[jira] [Updated] (SPARK-15568) TimSort and RadixSort can't support more than 1 billions elements

2016-05-26 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15568: --- Summary: TimSort and RadixSort can't support more than 1 billions elements (was: TimSort and RadixSo

[jira] [Created] (SPARK-15568) TimSort and RadixSort can't support more than 2 billions elements

2016-05-26 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15568: -- Summary: TimSort and RadixSort can't support more than 2 billions elements Key: SPARK-15568 URL: https://issues.apache.org/jira/browse/SPARK-15568 Project: Spark

[jira] [Commented] (SPARK-15554) Duplicated executors in Spark UI

2016-05-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301627#comment-15301627 ] Davies Liu commented on SPARK-15554: cc [~zsxwing] > Duplicated executors in Spark U

[jira] [Created] (SPARK-15554) Duplicated executors in Spark UI

2016-05-25 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15554: -- Summary: Duplicated executors in Spark UI Key: SPARK-15554 URL: https://issues.apache.org/jira/browse/SPARK-15554 Project: Spark Issue Type: Bug Compon

[jira] [Assigned] (SPARK-15332) OutOfMemory in TimSort

2016-05-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15332: -- Assignee: Davies Liu > OutOfMemory in TimSort > --- > > K

[jira] [Assigned] (SPARK-15391) Spark executor OOM during TimSort

2016-05-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15391: -- Assignee: Davies Liu > Spark executor OOM during TimSort > - >

[jira] [Resolved] (SPARK-12795) Whole stage codegen

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12795. Resolution: Fixed Fix Version/s: 2.0.0 > Whole stage codegen > --- > >

[jira] [Closed] (SPARK-14748) BoundReference should not set ExprCode.code to empty string

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-14748. -- Resolution: Won't Fix Won't fix this for now. > BoundReference should not set ExprCode.code to empty s

[jira] [Updated] (SPARK-12949) Support common expression elimination

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12949: --- Assignee: Liang-Chi Hsieh > Support common expression elimination > -

[jira] [Resolved] (SPARK-12949) Support common expression elimination

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12949. Resolution: Fixed Fix Version/s: 2.0.0 > Support common expression elimination > ---

[jira] [Commented] (SPARK-12949) Support common expression elimination

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15298537#comment-15298537 ] Davies Liu commented on SPARK-12949: Common subexpress elimination in Aggregate are s

[jira] [Updated] (SPARK-13135) Don't print expressions recursively in generated code

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13135: --- Assignee: Dongjoon Hyun > Don't print expressions recursively in generated code > ---

[jira] [Resolved] (SPARK-13135) Don't print expressions recursively in generated code

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13135. Resolution: Fixed Target Version/s: 2.0.0 > Don't print expressions recursively in genera

[jira] [Resolved] (SPARK-15433) PySpark core test should not use SerDe from PythonMLLibAPI

2016-05-24 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15433. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13214 [https://github.

[jira] [Updated] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

2016-05-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14343: --- Priority: Blocker (was: Critical) > Dataframe operations on a partitioned dataset (using partition d

[jira] [Commented] (SPARK-14946) Spark 2.0 vs 1.6.1 Query Time(out)

2016-05-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296697#comment-15296697 ] Davies Liu commented on SPARK-14946: [~raymond.honderd...@sizmek.com] Thanks for the

[jira] [Updated] (SPARK-15441) dataset outer join seems to return incorrect result

2016-05-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15441: --- Assignee: Wenchen Fan > dataset outer join seems to return incorrect result > ---

[jira] [Commented] (SPARK-15441) dataset outer join seems to return incorrect result

2016-05-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294807#comment-15294807 ] Davies Liu commented on SPARK-15441: How to we represent a null in Dataset? If it's a

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294785#comment-15294785 ] Davies Liu commented on SPARK-15285: [~kiszk] Go ahead, don't know why I can't assign

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15285: --- Assignee: (was: Wenchen Fan) > Generated SpecificSafeProjection.apply method grows beyond 64 KB >

[jira] [Resolved] (SPARK-15078) Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15078. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13188 [https://github.

<    5   6   7   8   9   10   11   12   13   14   >