[jira] [Created] (SPARK-12287) Support UnsafeRow in MapPartitions/MapGroups/CoGroup

Davies Liu created SPARK-12287: -- Summary: Support UnsafeRow in MapPartitions/MapGroups/CoGroup Key: SPARK-12287 URL: https://issues.apache.org/jira/browse/SPARK-12287 Project: Spark Issue Type:

[jira] [Created] (SPARK-12295) Manage the memory used by window function

Davies Liu created SPARK-12295: -- Summary: Manage the memory used by window function Key: SPARK-12295 URL: https://issues.apache.org/jira/browse/SPARK-12295 Project: Spark Issue Type:

[jira] [Created] (SPARK-12292) Support UnsafeRow in Generate

Davies Liu created SPARK-12292: -- Summary: Support UnsafeRow in Generate Key: SPARK-12292 URL: https://issues.apache.org/jira/browse/SPARK-12292 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-11885) UDAF may nondeterministically generate wrong results

[ https://issues.apache.org/jira/browse/SPARK-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11885: -- Assignee: Davies Liu (was: Yin Huai) > UDAF may nondeterministically generate wrong results

[jira] [Commented] (SPARK-11885) UDAF may nondeterministically generate wrong results

[ https://issues.apache.org/jira/browse/SPARK-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053528#comment-15053528 ] Davies Liu commented on SPARK-11885: The root cause is that we generate ExprId for ScalaUDAF in

[jira] [Resolved] (SPARK-11885) UDAF may nondeterministically generate wrong results

[ https://issues.apache.org/jira/browse/SPARK-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11885. Resolution: Fixed Fix Version/s: 1.5.3 > UDAF may nondeterministically generate wrong

[jira] [Resolved] (SPARK-11713) Initial RDD for updateStateByKey for pyspark

[ https://issues.apache.org/jira/browse/SPARK-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11713. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10082

[jira] [Assigned] (SPARK-12213) Query with only one distinct should not having on expand

[ https://issues.apache.org/jira/browse/SPARK-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12213: -- Assignee: Davies Liu > Query with only one distinct should not having on expand >

[jira] [Commented] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047186#comment-15047186 ] Davies Liu commented on SPARK-12179: Could you also test 1.6-RC1? I'm just wondering that the window

[jira] [Comment Edited] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047193#comment-15047193 ] Davies Liu edited comment on SPARK-12179 at 12/8/15 6:24 PM: - There are two

[jira] [Commented] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047193#comment-15047193 ] Davies Liu commented on SPARK-12179: There are two direction to narrow down the problem: 1) simplify

[jira] [Created] (SPARK-12213) Query with only one distinct should not having on expand

Davies Liu created SPARK-12213: -- Summary: Query with only one distinct should not having on expand Key: SPARK-12213 URL: https://issues.apache.org/jira/browse/SPARK-12213 Project: Spark Issue

[jira] [Resolved] (SPARK-12222) deserialize RoaringBitmap using Kryo serializer throw Buffer underflow exception

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-1. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull

[jira] [Updated] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12179: --- Priority: Critical (was: Minor) > Spark SQL get different result with the same code >

[jira] [Commented] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046014#comment-15046014 ] Davies Liu commented on SPARK-12179: This may be related to

[jira] [Resolved] (SPARK-12132) Cltr-C should clear current line in pyspark shell

[ https://issues.apache.org/jira/browse/SPARK-12132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12132. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull

[jira] [Resolved] (SPARK-12032) Filter can't be pushed down to correct Join because of bad order of Join

[ https://issues.apache.org/jira/browse/SPARK-12032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12032. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10073

[jira] [Resolved] (SPARK-12089) java.lang.NegativeArraySizeException when growing BufferHolder

[ https://issues.apache.org/jira/browse/SPARK-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12089. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull

[jira] [Created] (SPARK-12132) Cltr-C should clear current line in pyspark shell

Davies Liu created SPARK-12132: -- Summary: Cltr-C should clear current line in pyspark shell Key: SPARK-12132 URL: https://issues.apache.org/jira/browse/SPARK-12132 Project: Spark Issue Type:

[jira] [Updated] (SPARK-12089) java.lang.NegativeArraySizeException when growing BufferHolder

[ https://issues.apache.org/jira/browse/SPARK-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12089: --- Priority: Critical (was: Major) > java.lang.NegativeArraySizeException when growing BufferHolder >

[jira] [Commented] (SPARK-12089) java.lang.NegativeArraySizeException when growing BufferHolder

[ https://issues.apache.org/jira/browse/SPARK-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036194#comment-15036194 ] Davies Liu commented on SPARK-12089: Is it possible that you have a record larger than 1G? I don't

[jira] [Commented] (SPARK-12089) java.lang.NegativeArraySizeException when growing BufferHolder

[ https://issues.apache.org/jira/browse/SPARK-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036295#comment-15036295 ] Davies Liu commented on SPARK-12089: [~tyro89] Are you build a large Array using group by? How is the

[jira] [Commented] (SPARK-12089) java.lang.NegativeArraySizeException when growing BufferHolder

[ https://issues.apache.org/jira/browse/SPARK-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036280#comment-15036280 ] Davies Liu commented on SPARK-12089: Could you turn on debug log, and paste the java source code of

[jira] [Commented] (SPARK-12089) java.lang.NegativeArraySizeException when growing BufferHolder

[ https://issues.apache.org/jira/browse/SPARK-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036807#comment-15036807 ] Davies Liu commented on SPARK-12089: This query will not generate huge record, each record should be

[jira] [Updated] (SPARK-12089) [][]][][]][[[

[ https://issues.apache.org/jira/browse/SPARK-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12089: --- Summary: [][]][][]][[[ (was: java.lang.NegativeArraySizeException when growing BufferHolder) >

[jira] [Commented] (SPARK-12110) spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature Exception: ("You must build Spark with Hive

[ https://issues.apache.org/jira/browse/SPARK-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037349#comment-15037349 ] Davies Liu commented on SPARK-12110: [~aedwip] How do you launch the EC2 cluster using ec2/spark-ec2?

[jira] [Created] (SPARK-12077) Use more robust plan for single distinct aggregation

Davies Liu created SPARK-12077: -- Summary: Use more robust plan for single distinct aggregation Key: SPARK-12077 URL: https://issues.apache.org/jira/browse/SPARK-12077 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12077) Use more robust plan for single distinct aggregation

[ https://issues.apache.org/jira/browse/SPARK-12077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034620#comment-15034620 ] Davies Liu commented on SPARK-12077: https://github.com/apache/spark/pull/10075 > Use more robust

[jira] [Commented] (SPARK-12030) Incorrect results when aggregate joined data

[ https://issues.apache.org/jira/browse/SPARK-12030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034615#comment-15034615 ] Davies Liu commented on SPARK-12030: I also figured out the root cause last night, that's an

[jira] [Commented] (SPARK-6830) Memoize frequently queried vals in RDD, such as numPartitions, count etc.

[ https://issues.apache.org/jira/browse/SPARK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034635#comment-15034635 ] Davies Liu commented on SPARK-6830: --- +1 > Memoize frequently queried vals in RDD, such as

[jira] [Resolved] (SPARK-12090) Coalesce does not consider shuffle in PySpark

[ https://issues.apache.org/jira/browse/SPARK-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12090. Resolution: Fixed Fix Version/s: 1.6.0 1.5.3 2.0.0

[jira] [Assigned] (SPARK-12090) Coalesce does not consider shuffle in PySpark

[ https://issues.apache.org/jira/browse/SPARK-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12090: -- Assignee: Davies Liu > Coalesce does not consider shuffle in PySpark >

[jira] [Created] (SPARK-12090) Coalesce does not consider shuffle in PySpark

Davies Liu created SPARK-12090: -- Summary: Coalesce does not consider shuffle in PySpark Key: SPARK-12090 URL: https://issues.apache.org/jira/browse/SPARK-12090 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-11700) Memory leak at SparkContext jobProgressListener stageIdToData map

[ https://issues.apache.org/jira/browse/SPARK-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11700. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9990

[jira] [Commented] (SPARK-12032) Filter can't be pushed down to correct Join because of bad order of Join

[ https://issues.apache.org/jira/browse/SPARK-12032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15032217#comment-15032217 ] Davies Liu commented on SPARK-12032: [~marmbrus] Do you have some idea how to fix this? I had not

[jira] [Created] (SPARK-12054) Consider nullable in codegen

Davies Liu created SPARK-12054: -- Summary: Consider nullable in codegen Key: SPARK-12054 URL: https://issues.apache.org/jira/browse/SPARK-12054 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-11982) Improve performance of CartesianProduct

[ https://issues.apache.org/jira/browse/SPARK-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11982. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 9969

[jira] [Commented] (SPARK-12030) Incorrect results when aggregate joined data

[ https://issues.apache.org/jira/browse/SPARK-12030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15032857#comment-15032857 ] Davies Liu commented on SPARK-12030: [~smilegator] Could you post the related PRs here? So we can

[jira] [Assigned] (SPARK-12032) Filter can't be pushed down to correct Join because of bad order of Join

[ https://issues.apache.org/jira/browse/SPARK-12032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12032: -- Assignee: Davies Liu > Filter can't be pushed down to correct Join because of bad order of

[jira] [Created] (SPARK-12032) Filter can't be pushed down to correct Join because of bad order of Join

Davies Liu created SPARK-12032: -- Summary: Filter can't be pushed down to correct Join because of bad order of Join Key: SPARK-12032 URL: https://issues.apache.org/jira/browse/SPARK-12032 Project: Spark

[jira] [Resolved] (SPARK-12028) [SQL] get_json_object is unable to return a correct result for null literals

[ https://issues.apache.org/jira/browse/SPARK-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12028. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull

[jira] [Resolved] (SPARK-11997) NPE when save a DataFrame as parquet and partitioned by long column

[ https://issues.apache.org/jira/browse/SPARK-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11997. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull

[jira] [Resolved] (SPARK-11973) Filter pushdown does not work with aggregation with alias

[ https://issues.apache.org/jira/browse/SPARK-11973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11973. Resolution: Fixed Fix Version/s: 1.6.0 > Filter pushdown does not work with aggregation

[jira] [Created] (SPARK-12003) Expanded star should use field name as column name

Davies Liu created SPARK-12003: -- Summary: Expanded star should use field name as column name Key: SPARK-12003 URL: https://issues.apache.org/jira/browse/SPARK-12003 Project: Spark Issue Type:

[jira] [Updated] (SPARK-11997) NPE when save a DataFrame as parquet and partitioned by long column

[ https://issues.apache.org/jira/browse/SPARK-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11997: --- Priority: Blocker (was: Critical) > NPE when save a DataFrame as parquet and partitioned by long

[jira] [Commented] (SPARK-11997) NPE when save a DataFrame as parquet and partitioned by long column

[ https://issues.apache.org/jira/browse/SPARK-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15027516#comment-15027516 ] Davies Liu commented on SPARK-11997: It works well on 1.5 > NPE when save a DataFrame as parquet and

[jira] [Created] (SPARK-11997) NPE when save a DataFrame as parquet and partitioned by long column

Davies Liu created SPARK-11997: -- Summary: NPE when save a DataFrame as parquet and partitioned by long column Key: SPARK-11997 URL: https://issues.apache.org/jira/browse/SPARK-11997 Project: Spark

[jira] [Updated] (SPARK-11997) NPE when save a DataFrame as parquet and partitioned by long column

[ https://issues.apache.org/jira/browse/SPARK-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11997: --- Priority: Critical (was: Major) > NPE when save a DataFrame as parquet and partitioned by long

[jira] [Resolved] (SPARK-11969) SQL UI does not work with PySpark

[ https://issues.apache.org/jira/browse/SPARK-11969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11969. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull

[jira] [Resolved] (SPARK-12003) Expanded star should use field name as column name

[ https://issues.apache.org/jira/browse/SPARK-12003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12003. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull

[jira] [Assigned] (SPARK-12003) Expanded star should use field name as column name

[ https://issues.apache.org/jira/browse/SPARK-12003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12003: -- Assignee: Davies Liu > Expanded star should use field name as column name >

[jira] [Assigned] (SPARK-11982) Improve performance of CartesianProduct

[ https://issues.apache.org/jira/browse/SPARK-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11982: -- Assignee: Davies Liu > Improve performance of CartesianProduct >

[jira] [Assigned] (SPARK-11700) Memory leak at SparkContext jobProgressListener stageIdToData map

[ https://issues.apache.org/jira/browse/SPARK-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11700: -- Assignee: Davies Liu (was: Shixiong Zhu) > Memory leak at SparkContext jobProgressListener

[jira] [Created] (SPARK-11969) SQL UI does not work with PySpark

Davies Liu created SPARK-11969: -- Summary: SQL UI does not work with PySpark Key: SPARK-11969 URL: https://issues.apache.org/jira/browse/SPARK-11969 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-11982) Improve performance of CartesianProduct

Davies Liu created SPARK-11982: -- Summary: Improve performance of CartesianProduct Key: SPARK-11982 URL: https://issues.apache.org/jira/browse/SPARK-11982 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-11836) Register a Python function creates a new SQLContext

[ https://issues.apache.org/jira/browse/SPARK-11836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11836: -- Assignee: Davies Liu > Register a Python function creates a new SQLContext >

[jira] [Commented] (SPARK-10538) java.lang.NegativeArraySizeException during join

[ https://issues.apache.org/jira/browse/SPARK-10538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022726#comment-15022726 ] Davies Liu commented on SPARK-10538: I think we can re-open this once you find a way to reproduce the

[jira] [Created] (SPARK-11883) New Parquet reader generate wrong result

Davies Liu created SPARK-11883: -- Summary: New Parquet reader generate wrong result Key: SPARK-11883 URL: https://issues.apache.org/jira/browse/SPARK-11883 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11700) Memory leak at SparkContext jobProgressListener stageIdToData map

[ https://issues.apache.org/jira/browse/SPARK-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15018469#comment-15018469 ] Davies Liu commented on SPARK-11700: So there is at most one SQLContext leak per thread, I think it's

[jira] [Updated] (SPARK-11783) When deployed against remote Hive metastore, HiveContext.executionHive points to wrong metastore

[ https://issues.apache.org/jira/browse/SPARK-11783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11783: --- Priority: Critical (was: Blocker) > When deployed against remote Hive metastore,

[jira] [Resolved] (SPARK-10567) Reducer locality follow-up for Spark 1.6

[ https://issues.apache.org/jira/browse/SPARK-10567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10567. Resolution: Fixed Assignee: Matei Zaharia Fix Version/s: 1.6.0 > Reducer locality

[jira] [Commented] (SPARK-11855) Catalyst breaks backwards compatibility in branch-1.6

[ https://issues.apache.org/jira/browse/SPARK-11855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014135#comment-15014135 ] Davies Liu commented on SPARK-11855: cc [~marmbrus] > Catalyst breaks backwards compatibility in

[jira] [Updated] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11016: --- Assignee: (was: Davies Liu) > Spark fails when running with a task that requires a more recent

[jira] [Commented] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014055#comment-15014055 ] Davies Liu commented on SPARK-11016: [~sowen] I tried to assigned this to [~drcrallen] (who did the

[jira] [Updated] (SPARK-9506) DataFrames Postgresql JDBC unable to support most of the Postgresql's Data Type

[ https://issues.apache.org/jira/browse/SPARK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9506: -- Priority: Major (was: Blocker) > DataFrames Postgresql JDBC unable to support most of the Postgresql's

[jira] [Updated] (SPARK-9278) DataFrameWriter.insertInto inserts incorrect data

[ https://issues.apache.org/jira/browse/SPARK-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9278: -- Assignee: Cheng Lian > DataFrameWriter.insertInto inserts incorrect data >

[jira] [Commented] (SPARK-11850) Spark StdDev/Variance defaults are incompatible with Hive

[ https://issues.apache.org/jira/browse/SPARK-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014142#comment-15014142 ] Davies Liu commented on SPARK-11850: [~hvanhovell] This is on purpose, we had a long discussion about

[jira] [Updated] (SPARK-9686) Spark Thrift server doesn't return correct JDBC metadata

[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9686: -- Priority: Critical (was: Blocker) > Spark Thrift server doesn't return correct JDBC metadata >

[jira] [Commented] (SPARK-10567) Reducer locality follow-up for Spark 1.6

[ https://issues.apache.org/jira/browse/SPARK-10567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014125#comment-15014125 ] Davies Liu commented on SPARK-10567: Since https://issues.apache.org/jira/browse/SPARK-9852 is

[jira] [Resolved] (SPARK-9604) Unsafe ArrayData and MapData is very very slow

[ https://issues.apache.org/jira/browse/SPARK-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9604. --- Resolution: Fixed > Unsafe ArrayData and MapData is very very slow >

[jira] [Closed] (SPARK-11850) Spark StdDev/Variance defaults are incompatible with Hive

[ https://issues.apache.org/jira/browse/SPARK-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-11850. -- Resolution: Not A Problem > Spark StdDev/Variance defaults are incompatible with Hive >

[jira] [Commented] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014078#comment-15014078 ] Davies Liu commented on SPARK-11016: Thanks! > Spark fails when running with a task that requires a

[jira] [Commented] (SPARK-9506) DataFrames Postgresql JDBC unable to support most of the Postgresql's Data Type

[ https://issues.apache.org/jira/browse/SPARK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014100#comment-15014100 ] Davies Liu commented on SPARK-9506: --- [~cloud_fan] Could this be workaround by using a customized dialect

[jira] [Commented] (SPARK-9271) Concurrency bug triggered by partition predicate push-down

[ https://issues.apache.org/jira/browse/SPARK-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014128#comment-15014128 ] Davies Liu commented on SPARK-9271: --- [~lian cheng] Is this still a problem? > Concurrency bug triggered

[jira] [Updated] (SPARK-11785) When deployed against remote Hive metastore with lower versions, JDBC metadata calls throws exception

[ https://issues.apache.org/jira/browse/SPARK-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11785: --- Priority: Critical (was: Blocker) > When deployed against remote Hive metastore with lower

[jira] [Updated] (SPARK-11851) Unable to start spark thrift server against secured hive metastore(GSS initiate failed)

[ https://issues.apache.org/jira/browse/SPARK-11851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11851: --- Priority: Critical (was: Blocker) > Unable to start spark thrift server against secured hive

[jira] [Created] (SPARK-11864) Improve performance of max/min

Davies Liu created SPARK-11864: -- Summary: Improve performance of max/min Key: SPARK-11864 URL: https://issues.apache.org/jira/browse/SPARK-11864 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-11804) Exception raise when using Jdbc predicates option in PySpark

[ https://issues.apache.org/jira/browse/SPARK-11804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11804. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9791

[jira] [Resolved] (SPARK-11657) Bad Dataframe data read from parquet

[ https://issues.apache.org/jira/browse/SPARK-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11657. Resolution: Fixed > Bad Dataframe data read from parquet > >

[jira] [Updated] (SPARK-11657) Bad Dataframe data read from parquet

[ https://issues.apache.org/jira/browse/SPARK-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11657: --- Fix Version/s: 1.6.0 1.5.3 > Bad Dataframe data read from parquet >

[jira] [Assigned] (SPARK-11657) Bad Dataframe data read from parquet

[ https://issues.apache.org/jira/browse/SPARK-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11657: -- Assignee: Davies Liu > Bad Dataframe data read from parquet >

[jira] [Commented] (SPARK-11657) Bad Dataframe data read from parquet

[ https://issues.apache.org/jira/browse/SPARK-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012359#comment-15012359 ] Davies Liu commented on SPARK-11657: [~virgilp] Are you using Kyro? it's may be related to this one :

[jira] [Resolved] (SPARK-11767) Easy to OOM when cache large column

[ https://issues.apache.org/jira/browse/SPARK-11767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11767. Resolution: Fixed Fix Version/s: 1.6.0 > Easy to OOM when cache large column >

[jira] [Resolved] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11016. Resolution: Fixed Issue resolved by pull request 9748 [https://github.com/apache/spark/pull/9748]

[jira] [Resolved] (SPARK-11583) Make MapStatus use less memory uage

[ https://issues.apache.org/jira/browse/SPARK-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11583. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9746

[jira] [Created] (SPARK-11805) SpillableIterator should free the in-memory sorter while spilling

Davies Liu created SPARK-11805: -- Summary: SpillableIterator should free the in-memory sorter while spilling Key: SPARK-11805 URL: https://issues.apache.org/jira/browse/SPARK-11805 Project: Spark

[jira] [Resolved] (SPARK-11737) String may not be serialized correctly with Kyro

[ https://issues.apache.org/jira/browse/SPARK-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11737. Resolution: Fixed Fix Version/s: 1.5.2 1.6.0 Issue resolved by pull

[jira] [Updated] (SPARK-11737) String may not be serialized correctly with Kyro

[ https://issues.apache.org/jira/browse/SPARK-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11737: --- Fix Version/s: (was: 1.5.2) 1.5.3 > String may not be serialized correctly

[jira] [Resolved] (SPARK-11643) inserting date with leading zero inserts null example '0001-12-10'

[ https://issues.apache.org/jira/browse/SPARK-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11643. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9701

[jira] [Updated] (SPARK-11752) fix timezone problem for DateTimeUtils.getSeconds

[ https://issues.apache.org/jira/browse/SPARK-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11752: --- Fix Version/s: (was: 1.5.2) 1.5.3 > fix timezone problem for

[jira] [Resolved] (SPARK-11743) Add UserDefinedType support to RowEncoder

[ https://issues.apache.org/jira/browse/SPARK-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11743. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9712

[jira] [Resolved] (SPARK-11752) fix timezone problem for DateTimeUtils.getSeconds

[ https://issues.apache.org/jira/browse/SPARK-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11752. Resolution: Fixed Fix Version/s: 1.5.2 1.6.0 Issue resolved by pull

[jira] [Reopened] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reopened SPARK-11016: Assignee: (was: Liang-Chi Hsieh) https://github.com/apache/spark/pull/9243 is reverted >

[jira] [Commented] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007565#comment-15007565 ] Davies Liu commented on SPARK-11016: [~charles.al...@acxiom.com] Could you send your patch to

[jira] [Reopened] (SPARK-11271) MapStatus too large for driver

[ https://issues.apache.org/jira/browse/SPARK-11271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reopened SPARK-11271: https://github.com/apache/spark/pull/9243 is reverted > MapStatus too large for driver >

[jira] [Closed] (SPARK-11271) MapStatus too large for driver

[ https://issues.apache.org/jira/browse/SPARK-11271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-11271. -- Resolution: Duplicate Assignee: (was: Liang-Chi Hsieh) Fix Version/s: (was:

[jira] [Assigned] (SPARK-11767) Easy to OOM when cache large column

[ https://issues.apache.org/jira/browse/SPARK-11767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11767: -- Assignee: Davies Liu > Easy to OOM when cache large column >

[jira] [Created] (SPARK-11767) Easy to OOM when cache large column

Davies Liu created SPARK-11767: -- Summary: Easy to OOM when cache large column Key: SPARK-11767 URL: https://issues.apache.org/jira/browse/SPARK-11767 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-11643) inserting date with leading zero inserts null example '0001-12-10'

[ https://issues.apache.org/jira/browse/SPARK-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11643: -- Assignee: Davies Liu > inserting date with leading zero inserts null example '0001-12-10' >

[jira] [Commented] (SPARK-10712) JVM crashes with spark.sql.tungsten.enabled = true

[ https://issues.apache.org/jira/browse/SPARK-10712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15004636#comment-15004636 ] Davies Liu commented on SPARK-10712: How is you small table looks like? Does 1.5.2-RC2 still have

<    6   7   8   9   10   11   12   13   14   15   >